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ABSTRACT 

SOME  RECENT  CONTRIBUTIONS  'rn  COMPUTER  PROGRAMMING  MANAGEMENT 

The  System  Development  Corporation  (SDC)  has  been  conducting  research  into  the 
management  of  the  computer  programming  process  since  1963.  This  paper  briefly 
describes  some  of  the  results  of  this  re  rch,  plus  individual  research  by  the 
author  ,  and  the  relevance  and  usefulness  of  the  results  to  the  operating 

manager . 

The  paper  emphasizes  four  specific  types  of  management  tools  that  were  produced: 

(1)  planning  aids;  (2)  cost  estimating  guides;  (3)  a  project  reporting  and 
control  ays t ’.in;  (4)  a  technique  for  ev  uating  the  effectiveness  of  certain 
classes  of  computer-centered  information  systems  Each  of  these  tools  is 
briefly  described,  and  the  research  design  and  procedures  used  in  their  develop¬ 
ment  are  mentioned.  The  experience  gained  —  at  SDC  and  elsewhere  —  in 
applying  traditional  research  techniques  to  the  computer  programming  process 
has  yielded  certain  insights  regarding  the  economic  and  management  dimensions 
of  computer  programming,  and  several  of  these  insights  are  discussed.  Foremost 
among  this  author's  conclusions  is  that  the  management  principles  that  apply  to 
any  other  coordinative  activity  are  equally  applicable  to  the  computer  program¬ 
ming  process,  ipaciflc  techniques  may  differ,  especially  at  the  lower  echelons, 
but  these  differences  pertain  to  the  technical  skills  and  procedures  -f  the 
production  process;  they  are  engineering,  not  management  or  economic  issues. 

The  paper  concludes  with  several  general  observations  pertinent  to  future  research 
in  computer  programing  economics  and  management,  and  the  use  of  the  management 


tools  described. 
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SOME  RECENT  C0NT4 IBUTIONS 
TO  CuMPUTER  PROGRAMMING  MANAGEMENT 

INTRODUCTION 

This  paper  will  discuss  four  comparatively  recent  tools  that  can  be  useful 
to  the  manager  of  computer  programming:  (1)  a  planning  aid;  (2)  st 
estimating  guides;  (3)  a  project  reporting  and  control  system;  and  (4)  a 
technique  for  evaluating  the  effectiveness  of  certain  classes  of  computer- 
oriented  irformation  systems.  The  contribution  of  general  management 
principles,  and  the  degree  to  which  they  shape  the  process  of  creating 
tools  or  techniques,  should  become  apparent  in  the  following  sections  of 
this  paper . 

Management  is  defined  here  as  the  process  or  accomplishing  objectives  by 
establishing  an  environment  favorable  to  performance  by  people  operating 
in  organized  groups.  The  essence  of  managing  consists  in  the  .^tainment 
of  coordination  or  harmony  of  individual  effort  toward  the  achievement  of 
group  goals.  The  management  process  may  be  said  to  consist  of  the  performance 
of  specific  functions,  namely:  planning,  organizing,  staffing  and  assembling 
resources,  direction,  and  control  (1).  Each  of  these  functions  applies  also 
to  the  management  of  projects  whose  products  are  computer  programs. 


«  <* 
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It  is  my  opinion  that  principles  of  management  exist  that  are  universal  to 
all  forms  of  orwir  dinative  activity.  These  principles  consist  of  the 
fundamental  causal  relationships  that  explain  and  help  predict  the  results 
of  the  management  process,  and  serve  to  improve  the  results  of  the  process 
in  terms  of  the  desired  goals.  Management  principles,  therefore,  apply 
equally  well  to  the  fighting  of  a  battle,  the  running  of  a  technical  meeting, 
or  the  creation  of  a  computer  program.  One  such  universal  management  principle, 
for  example,  has  been  called  the  "principle  of  the  primary  of  planning"  (1). 

That  is,  planning  is  the  primary  requisite  to  the  other  managerial  functions 
of  organizing,  staffing,  direction,  and  control.  This  means  that  the  degree 
of  control  over  a  programming  project  (regardless  of  whether  we  are  talking 
about  schedules,  ttrn-hours ,  or  other  resources)  can  be  no  greater  than  the 
extent  to  which  adequate  plans  have  been  made  for  the  project;  it  can  be 
less,  of  course,  since  contingencies  can  force  modifications  in  even  the  best 
laid  plans — but  the  extent  of  planning  sets  the  degree  of  control  that  is 
posslbLe.  I  suspect  that  this  is  the  primary  reason  why  control  is  lost 
on  many  computer  programming  projects.  It  is  not  the  comparative  newness 
of  the  computer  programming  process,  difficulties  with  programmers,  or 
technical  factors — it  is  simply  that  the  programming  projects  are  not 
acequately  planned  in  the  first  place. 

The  Implications  to  the  r  magers  of  computer  programming  of  the  universal 
applicability  of  management  principles  is  that  their  immediate  and  most 
productive  course  of  action  is  to  applv  the  principles  and  tools  already 
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at  hand.  The  magnitude  of  improvements  possible  with  this  approach  is 
illustrated  by  my  own  research  on  administrative  costs  in  the  life  insurance 
industry  (2),  which  showed  that  administrative  costs  could  be  at  least  halved 
for  the  average  life  insurance  company  using  their  existing  ADP  equipment, 
providing  that  these  companies  manage  their  affairs  so  as  to  achieve  the 
performance  already  demonstrated  by  their  peers.  The  implications  of  this 
universality  of  management  principles  to  researchers,  on  the  other  hand,  is 
that  these  principles  can  become  the  guidelines  for  useful  research.  To  the 
extent  that  management  principles  are  generally  applicable,  and  the  management 
lessons  of  other  disciplines  can  be  used  for  computer  programming  projects, 
the  comparative  newness  of  the  omputer  programming  process  becomes  largely 
irrelevant;  and  productive  research  in  the  economics  and  management  of  the 
programming  activity,  giver,  this  existing  foundation  of  management  principles, 
wii'  most  profitably  focus  on  the  creation  of  guides  and  devices  for  answering 
technical,  rather  than  managerial  questions.  This  process,  including  the 
research  emphasis  on  technological  factors,  U-  Illustrated  by  the  descriptions 


> 


that  follow. 
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A  PLANNING  AID 

As  in  other  activities,  planning  computer  programming  projects  involves  the 
selection  £rom  among  alternatives  of  future  courses  of  action.  To  promote 
accurate  planning  by  first- level  supervisors,  the  Office  of  Naval  Research 
sponsored  the  development  of  a  planning  guide  (3),  designed  to  stimulate  more 
complete  consideration  of  the  entire  computer  programming  process.  A  set  of 
planning  and  management  tasks  are  defined  in  terms  of  the  computer  program 
development  process;  this  Involved  dividing  the  programming  process  into 
distinct  phases,  or  steps,  and  identifying  the  detailed  tasks  (36  separate 
tasks  are  defined  in  the  guide)  within  each  Btep.  For  each  task,  the  information 
and  document  inputs  required  to  perform  the  task  were  listed,  suhtasks  (there 

m 

were  4  to  13  for  each  task  in  the  guide)  are  defined,  the  information  and 
documents  resulting  as  output  of  the  task  are  listed,  as  are  several  of  the  more 
Important  factors  that  affect  the  costs  of  performing  the  tasks.  A  sample  lay- 

■  f  •  .  y  " 

out  for  presenting  this  information  is  illustrated  in  Figure  1.  What  was  done 
in  the  construction  of  the  planning  guide,  in  essence,  was  to  study  the  technical 
process  of  computer  programming,  and  prepare  a  check-list  for  the  planner  to 
consider;  the  planner  can  then  more  readily  apply  planning  principles  with  which 
he  was  already  familiar  to  this  technical  process. 

It  is  noteworthy  that  the  steps  in  the  computer  programming  process,  although 
containing  technical  distinctions,  are  nevertheless  qulce  similar  to  those  in 
the  procurement  cf  other  systems.  This  is  Illustrated  in  Figure  2,  where  a 
comparison  is  made  between  steps  in  a  general  system  life  cycle.  The  four 
general  steps  in  Figure  1  are  substantially  equivalent  to  the  procurement 
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Figure  2.  Relationship  of  the  Computer  Program  Procurement 
Process  to  the.  Procurement  of  Other  Products. 
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phages  (conception,  definition,  acquisition,  operation)  specified  in  the  USAF 
system  management  process  (4),  which  was  developed  originally  for  the  procure¬ 
ment  of  weapons  systems  hardware;  in  fact,  the  universality  of  management 
principles  and  their  applicability  to  the  computer  programming  process  is  ag&ir 
by  the  efforts  expended  in  applying  the  USAF  systems  aanageasent 
the  management  o*  computer  programing  (5,  6,  7,  8,  9), 


illustrated 
concepts  to 
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CCXJT  ESTIMATING  GUIDES 

The  Electronic  Systems  Division  of  the  Air  Force  Systems  Command  has  sponsored 
work  directed  toward  the  analysis  of  experience  data  to  develop  tools  for 
estimating  the  costs  (man  hours,  computer  hours,  and  other  resources  expended) 
of  the  computer  pro granting  process.  One  of  these  studies,  by  the  bystem 
Development  Corporation,  collected  data  by  questionnaire  on  169  computer  pro¬ 
grams  oy  both  government  and  industry.  This  work  was  conducted  in  cycles  (10), 
each  marked  by  collection  and  analysis  of  new  data  to  improve  upon  earlier 
results,  and  culminated  in  a  handbook  (ill  that  also  included  a  collection  of 
estimating  rules -of-thumb  gleaned  from  an  examination  of  the  technical  literature. 
The  System  Development  Corporation  has  been  engaged  in  such  research  on  the 
economics  and  management  of  computer  programming  since  196,3*  H*e  second  study 
was  done  by  the  Planning  Research  Corporation,  and  involved  an  extensive 
interview  of  the  cognizant  personnel  who  worked  on  a  total,  of  eighteen  computer 
programs  (12).  Both  of  these  studies  used  standard  multiple  regression  techniques 
to  generate  equations  for  relating  required  resources  to  the  various  factors 
presumed  to  influence  the  magnitude  of  these  resources. 

The  major  portion  of  the  statistical  material  published  in  the  SDC  handbook  is 
based  on  data  covering  only  computer  program  design,  coding,  and  functional 
test  (steps  1,  3,  and  6)  of  the  computer  program  life  cycle  described  in 
Fif^ure  2.  These  data  were  divided  into  subsemples  based  on  several  types  of 
urograms,  cad  some  of  the  characteristics  of  this  subsample  data  —  in  terms 
of  the  three  oasic  resources  exnended  in  writing  programs  --  are  listed  in 
Table  1.  It  is,  interesting  to  note  that  in  terms  of  object  instructions 


RESOURCE  EXPENDITURE  RATE  PER  1000  OBJECT  INSTRUCTIONS 
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produced,  procedure-oriented  languages  (POL)  demonstrates  a  statistically 
significant  advantage  over  machine-oriented  languages  (MOL),  with  lower  resource 
expenditure  rates  for  both  man-months  and  computer  hours;  discussions  of  this 
point,  and  other  conclusions  from  the  data,  are  elaborated  elsewhere  (11,  13). 

The  data  in  Table  I  may  be  used  in  estimating  resources  if  a  reasonable 
estimate  of  the  number  of  object  instructions  can  also  be  made. 

Figure  3  based  on  a  different  arrangement  of  the  data,  is  also  useful  in  ' 

estimating  man-months  when  the  approximate  number  of  object  instructions  is 
known.  This  curve  was  developed  by  arranging  all  of  the  programs  in  the  sample 
in  ascending  order  by  production  rate  (man-months  per  1000  instructions),  and 

plotting  these  production  rates  for  each  program  against  the  accumulative  per-  | 

| 

cent  in  the  total  sample  covered  by  the  sequence.  Thus,  that  subset  of  programs  ^ 

that  comprises  the  kO $  of  the  total  sample  with  the  lowest  production  rates  I 

contains,  from  Figure  j,  production  rates  of  two  man-months  per  1000  object 

instructions  or  less.  This  construction  permits  the  abscissa  to  serve  as  ar 

overall  measure  of  the  difficulty  of  the  programming  job.  Example:  if  the 

est'.aator  subjectively  believed  the  program  to  be  estimated  is  more  "difficult" 

than  the  median  of  the  sample  (50^  on  the  abscissa),  but  not  as  "difficult"  as 

the  more  extreme  values,  he  might  choose  to  use  production  rates  for  the  60-b0 

percent  range;  then  the  expected  resource  expenditure  rate  taken  from  the 

ordinate  would  be  3.9  to  6.3  man-months  pci  1000  object  instructions.  In 

Figure  3,  the  typical  range  is  arbitrarily  defined  to  exclude  the  upper  and 

lower  20  percent ij.es „  The  high  slope  of  the  curve  within  this  typical  range  is 

an  indication  of  the  large  variation  in  production  rates  in  the  sample.  'This 
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variation  could  mean  that  there  are  many  factors,  including  intangibles,  that 
affect  the  expenditure  of  resources  in  computer  programming;  on  the  other  hand, 
it  could  also  indicate  that  accepted  management  principles  had  not  been 
vigorously  applied  to  the  production  process  of  the  programs  in  the  sample. 

Multiple  regression  techniques  hav-3  also  been  used  to  investigate  the  impact 
of  various  parameters  on  the  expenditure  of  resources  in  computer  , "ogramming. 
Figure  4  illustrates  one  of  the  outputs  of  these  efforts.  Such  work  not  only 
produces  equations,  as  in  Figure  4,  that  can  be  used  for  estimating  purposes 
(with  appropriate  caveats);  it  also  serves  to  identify  factors  that  have 
statistical!  significant  impact  on  expenditures,  and  hence  directs  .  anagement 

»  attention  to  those  critical  factors,  many  of  which  are  subject  to  manager  ent 

control.  A  disadvantage  also  arises,  however,  when  the  results  of  cost  \-esearch 

* 

are  published  in  the  form  of  equations,  as  in  Figure  4.  Such  equations  are 
frequently  interpreted  by  the  user  as  demonstrating  a  specific  causative 
relationship;  this  is  not  the  case.  Equations  developed  by  multiple  regression 
techniques  do  reveal  important  parameters,  and  may  represent,  those  relationships 
that  provide  the  mo^t  statistically  significant  manner  of  describing  the 
character  of  the  analyst's  sample;  however,  they  do  not  necessarily  represent 
natural  laws,  as  do  many  of  the  equations  used  by  the  engineer  or  physicist. 
Alto,  they  must  be  used  in  their  entirety  or  not  it.  all;  if  a  value  for  any 
one  of  the  independent  variables  in  Figure  4  were  not  available,  the  equation 
could  not  be  used  an  it  stands,  since  repeating  the  multiple  regression 
analysis  w.thout  the  missing  parameter  would  result  in  a  reassignment  of 
weights  to  all  of  the  remaining  variables  and  the  Y-intercept  of  V.:n  equation. 
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1MTA  BASE: 

LARGE  COMPUTER  SUBSAMPLE  (consisting  of  systems 

WHOSE  PURCHASE  PRICE  IS  AT  LEAST  $750,000.) 

TOTAL  MAN  MONTHS  -  Yj  .  WHERE: 


N  -  106  COMPUTER  PROGRAMS 

aE  ”  22 
t  -  .83 
r2  -  ,68 
MEAN  -  54 
STD.  DEV.  -  71 


Yi  “  +  .049  +  13.2X0  -  .236X25  *  ,52%  +  4.5%  +  .09% 

-F.%i  +25.1X51  +  22,!%  +  26.%  -.25% 

"  +  I0.% 


-  Complexity  ~A  Program  Syaten  Interface.  Coded:  more  than 
502  of  deaign  effort  devoted  to  data  tranafer  problem*  to 

or  from  the  program  data  point  -  2;  betwu  n  102  and  502  efforc 
to  data  tranafer  problem*  *  1;  ieaa  than  102  *  O. 

-  Percent  Clerical  Instructions.  Coded  In  percent. 

-  Percent  Information  Storage  and  Retrieval  Functions.  Coded 
In  percent . 

-  Frequency  of  Operation.  Coded:  not  applicable  •  0;  lest 
than  1/month,  more  than  1/month  and  less  than  i/ueek  -  2; 
more  than  1/week  and  leas  than  1/day  •  3;  dally  »  4;  utility 
c  on-line  (lncludl-q  compilers)  -  5. 

-  External  Documentation.  Coded:  number  of  pages  written  for, 
or  distributed  to,  customers. 

-  Business  Coded:  as  mutually  exclusive  binary  variables; 

l.e.,  programs  classified  as  business  application 
-  1;  remaining  applications  •  0. 

-  First  Program  on  Computer:  Cded:  ye*  «  1;  no  ■  0. 

-  Special  Display  Equipment:  Coded:  special  display 
equlpme"t  used  *  1;  not  used  •  0. 

-  Random  Acce**  Device  Used.  Coded:  use  of  such  storage 
•  t ;  such  storage  not  used  -  0. 

-  Percent  Programmers  Participating  In  Program  Design. 

-  Personnel  Continuity.  Coded:  number  ot  nersonn*l  working  for 
Che  duration  of  the  project,  divided  by  the  maximum  numbar 
assigned  «t  any  ona  time. 

-  Numbar  of  Locations  for  Program  Lata  Pol'?  Development. 


Note:  Subscripts  are  thoaa  uaad  In  original  source  document  (11). 


Figure  4.  Equation  for  Estimating  Man  Months 

for  Programs  Developed  on  Large  Computer* 
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Another  application  of  the  available  computer  programming  cost  data  is  illus¬ 
trated  in  Figure  5.  Here,  expenditures  of  man-months  are  related  to  the 
concomitant  expenditures  in  computer  time  required  to  debug  the  programs  in 
the  sample  The  comparative  advantage  of  the  POLs  in  this  sample  of  programs 
is  also  apparent  in  these  relationships.  The  relationships  in  Figure  5,  as  in 
the  equation  in  Figure  L,  again  do  not  necessarily  represent  cause  and  effect; 
that  is,  it  is  not  meant  to  imply  that  a  given  expenditure  of  man-months  will 
require  a  concomitant  indicated  expenditure  of  computer  hours.  However,  to 
the  extent  that  the  sample  used  in  this  study  is  representative  of  a  computer 
program  to  be  written,  the  estimator  who  has  already  determined  what  his 
expected  man-months  will  be  can  also  arrive  at  an  estimate  of  computer  hours, 
using  Figure  5;  to  this  extent,  Figure  5  can  be  a  useful  portrayal  of  the 
historical  data. 

Again,  the  above  development  of  empirical  tools  to  aid  in  the  estimation  of 
computer  programming  costs  involved  nothing  new  in  the  way  of  either  principles 
or  techniques;  it  did  involve  an  analysis  of  the  components  of  the  computer 
programming  process,  and  the  application  of  available  methodology  to  the  study 
of  this  process.  And  as  is  coonon  in  empirical  cost  research  in  other  fields, 
the  most  significant  limitation  of  these  studies  centered  around  the  collection 
of  adequate  cost  data.  It  was  this  recognition  of  the  inadequacies  of  ex-post¬ 
fact  o  data  used  in  all  of  these  studies,  as  well  as  the  possibilities  offered 
for  more  direct  management  control  of  computer  programming  projects,  that 
shifted  efforts  at  5 DC  toward  the  development  of  a  cost  collection  and 
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control  svstem.  If  resources  could  be  measured  as  they  were  expended,  a  mere 
accurate  cost  history  of  computer  programming  projects  could  be  compiled  for 


jse  in  future  research. 
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A  COMPUTER  PROGRAMMING  PROJECT  REPORTING  SYSTEM 
The  Control  Proem 

The  managerial  function  of  control  Involves  the  measurement  and  correction  of 
the  performance  of  subordinates  In  order  to  make  certain  that  enterprise 
objectives  and  the  plana  devised  to  attain  them  are  accomplished  (l).  Control, 
therefore.  Implies  the  existence  of  plans,  and  a  fundamental  principle  of  con¬ 
trol  applicable  at  any  level  of  the  organization  is  that  controls  can  be  no 
more  effective  than  the  degree  to  which  they  reflect  the  character,  structure, 
and  degree  of  detail  inherent  in  these  plans.  Thus,  if  we  are  going  to  control 
any  coordinated  human  effort,  including  computer  programming,  we  must  be 
willing  to  Invest  the  effort  required  to  .produce  an  adequate  plan.  Perhaps  the 
most  significant  contribution  to  management  made  by  the  network  techniques  such 
as  PERT  and  CPM  lies  not  in  their  capabilities  of  reporting  deviations  or 
"critical  paths"  expeditiously,  although  these  are  valuable  attributes,  but 
rather  in  the  fact  that  they  force  managers  to  plan.  To  use  PERT,  tasks  and 
milestones  must  bo  defined,  interrelationships  of  these  tasks  and  milestones 
established,  and  schedules  (and  often  resources)  estimated.  These  comments 
suggest  that  one  of  the  primat7  reasons  why  programming  projects  have  slipped 
schedules  and  grossly  overrun  their  budgets  in  that  they  were  not  adequately 
planned  in  the  first  place,  and  hence  adequate  control  simply  was  not  possible. 

Programming  Project  Reporting  and  Control 

In  recent  years,  attention  has  been  given  to  planning  and  control  of  computer 
programming  by  commercial  computer  users  (e.g.,  14),  computer  manufacturers 
(e.g.,  15);  agencies  of  the  Department  of  Defense  (e.g.,  5);  and 
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professional  and  management  organizations  (l6).  The  Electronic  Systems 
Division  of  the  USAF  Systems  Command  has  sponsored  work  at  the  System  Develop¬ 
ment  Corporation  for  the  development  of  a  programming  project  reporting  system 
intended  for  use  by  Air  Force  agencies.  There  were  two  objectives  of  this 
reporting  system: 

1.  To  provide  a  vehicle  for  the  planning  and  control  of  USAF  computer 
programming  projects. 

2.  To  collect  a  data  base,  from  an  analysis  of  which  a  better  understanding 
of  the  factors  affecting  computer  programming  could  emerge.  This 
Included  the  potential  for  developing  more  accurate  resource  estimating 
relationships . 

The  resulting  product  (17)  consisted  of  a  system  wherein  a  computer  programming 
project  was  divided  into  the  nine  steps  illustrated  in  Figure  2.  These  steps 
were  further  subdivided  into  tasks  (e.g.,  Step  8— Information  Processing 
Installation  and  Implementation — consists  of  such  tasks  as  file  conversion, 
operational  testing,  preparation  of  operating  manuals,  training,  coordination, 
etc.)  for  use  with  larger  projects.  The  end  points  of  these  activities  (steps 
and  tasks)  thus  provide  milestones  for  planning.  And  if  all  personnel  working 
on  a  project  periodically  report  the  time  and  resources  they  spend  on  each  step 
or  tasks,  progress  can  be  effectively  compared  with  plans  and  variances,  if  any, 
discovered  so  that  corrective  action  may  be  taken. 

Figure  6  illustrates  a  sample  summary  report  that  would  result  from  the  use  of 
this  reporting  system.  For  the  sample  project  shown,  only  the  completion  of 


Figure  6.  A  Sample  Project  Summary  Report 
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the  nine  basic  steps  were  differentiated  as  milestones  in  tne  original  plan. 

In  this  illustration,  Step  4  has  been  completed,  but  the  project  is  behind 
(with  an  estimated  3  day  slippage  for  the  completion  of  Step  5),  with  a  current 
total  overrun  to  date  of  23  man  hours  and  1.82  computer  hours;  if  the  overrun 
continues  at  the  rate  experienced  to  date,  a  total  overrun  of  37  man  hours  is 
expected . 

This  projec.  reporting  system  requires  as  input  data  the  original  estimates  by 
step  (and  by  task  if  this  degree  of  control  is  desired) ,  periodic  actual 
expenditures,  revised  estimates,  and  comments  if  any.  Outputs  such  as  Figure  6 
can  then  be  prepared  either  manually  or  by  a  computer.  The  value  of  such  a 
system  as  an  aid  in  planning  and  control  should  be  evident  from  an  observation 
of  Figure  6.  The  value  of  the  system  in  developing  a  data  base  for  future 
research  on  t  -  economics  of  the  computer  programming  process  is  predicated  to 
a  large  extent  on  how  well  comparability  between  various  projects  can  be  main¬ 
tained;  adequate  comparability  can  be  achieved  if  at  least  the  nine  suggested  steps 
are  used  consistently  for  the  planning  of  all  projects,  if  a  standard  set  of 
subtasks  is  consistently  used  whenever  more  detailed  control  is  justified,  and 
if  some  additional  descriptive  data  (e.g.  ,  type  of  application,  language  used, 
machine  used,  etc..)  is  collected  on  each  project  (l8). 


I* 
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A.  TECHNIQUE  FOR  EVALUATING  SYSTEM  EFFECTIVENESS 

The  previous  material  in  this  paper  dealt  directly  with  planning,  estimating, 
and  controlling  the  computer  programming  process.  The  tools  presented  are 
intended  for  use  by  operating  managers  of  computer  programming  projects.  By 
contrast,  the  following  material  is  far  more  conceptual  and  abstract.  The 
purpose  is  to  briefly  present  the  principal  elements  of  a  technique  for 
evaluating  a  total  productive  system  of  which  operating  personnel,  computers, 
and  computer  programs  all  play  a  part.  This  material  is  thus  intended  more 
for  the  staff  specialist,  whose  interest  is  in  the  creation  of  management  tools 
and  management  information  systems.  Some  of  the  more  detailed  operational 
considerations  of  the  application  of  the  proposed  technique,  with  a  discussion 
of  the  results  of  a  trial  using  data  from  the  life  insurance  industry,  are 
discussed  elsewhere  (2,  19);  many  of  the  detailed  procedures  necessary  to  adapt 
the  techniques  specifically  in  computer  programming  management  problems  still 
remain  to  be  developed. 

The  need  for  criteria  and  a  methodology  for  evaluating  the  design  and  performance 
of  ADP  systems  has  been  frequently  mentioned  in  the  current  literature  (20).  Of 
particulat  interest  for  this  paper  is  the  call  for  measures  in  the  form  of 
indices  (21),  since  this  is  specifically  achieved  with  the  technique  that  follows. 
Index  numbers  are  devices  for  measuring  differences  in  the  magnitude  of  a  proup 
of  related  variables  (22)  and  are  particularly  useful  for  such  complex  phenomena 
as  the  general  price  level  (e.g.,  the  Bureau  of  Labor  Statistics  Index  of 
Wholesale  Commodity  Prices),  business  activity  (e.g.,  the  Federal  Reserve  Index 
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of  Industrial  Production),  or  qualitative  changes  or  differences  (e.g.,  Storie's 
Index  for  Rating  the  Agricultural  Value  of  Soils). 

Th^re  has  also  been  a  widely  expressed  (e.g.,  Mr.  Brandon's  paper  in  this 
session)  need  for  standards  of  quality,  or  a  means  for  evaluating  the  effective¬ 
ness  of  computer  programs.  The  principal  difficulty  in  achieving  adequate 
measures  of  a  computer  program’s  effectiveness,  however,  is  the  inescapable 
fact  that  computer  programs  themselves  are  almost  never  an  end  product;  rather, 
computer  programs  are  the  means  by  which  computers  are  used  to  achieve  other 
purposes.  Thus,  a  measurement  that  focuses  directly  on  computer  program 
efficiency  or  effectiveness  constitutes,  by  definition,  a  sub-optimization. 

This  is  why  such  measures  as  compile  time,  throughput  time,  amount  of  core  used, 
average  process  time  per  run,  etc.,  will  never  be  entirely  satisfactory,  even 
if  conceptual  problems  such  as  the  definition  of  a  "typical"  job  mix  or 
benchmark  problem  could  be  resolved. 

The  technique  espoused  herein  avoids  a  direct,  focus  on  either  the  computer  hard¬ 
ware  or  the  programs  by  which  it  operates.  Instead,  overall  measures  of  the 
productive  process  are  provided,  along  with  a  means  for  tracing  the  components 
of  these  measures  to  their  origins.  The  value  of  computer  programs  is  thus 
derived  by  implication  from  their  effects  erformance  of  the  total 

system  of  which  computer  programs  and  hardware  are  but  u  part. 

Ine  object  of  the  material  to  follow  is  to  describe  the  proposed  method  for 
developing  evaluation  indices  for  measuring  relative  overall  operating  efficiency 
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of  administrative  systems  augmented  by  ADP.*  This  method  is  applicable  to 
those  systems  where,  for  the  total  system: 

1.  An  objective,  numerical  measure  of  the  system’s  output 
can  be  devised, 

2.  Data  from  a  sample  of  organizations  of  generally  similar 
outputs  can  be  obtained. 

3.  More  than  one  input  factor  (e.g.,  personnel,  and  computer 
hardware;  is  important  to  the  productive  process. 


The  method  borrows  from  some  well  recognized  tools  of  the  economist,  part¬ 
icularly  the  concept  of  the  production  function.  The  idea  behind  the  production 
function  is  that  the  physical  volume  of  output  depends  on  the  quantities  of 
productive  agents  used  in  the  production  process,  and  the  efficiency  with  which 
tiiey  are  used.  Although  we  will  direct  attention  to  only  two  productive  agents, 
Jabot  and  capital  (or,  more  specifically,  manpower  and  EDP  equipment),  it  is 
possible  to  extend  the  method  to  as  many  productive  agents  as  desired. 


*  Efficiency  is  defined  here  as  the  attainment  of  objectives  at  the  least  ex¬ 
penditure  of  resources-  or,  as  Harrington  Emerson  phrased  it,  efficiency  is 
"...the  relation  between  what  is  accomplished  and  what  might  be  accomplished. 

(23) 


An  administrative  system  is  defined  as  a  productive  operation  whose  function 
consists  essentially  of  processing  information  or  data,  and  does  not  involve 
the  physical  handling  of  goods  or  materials.  Administrative  systems  are  a 
creation  of  management,  a  tool  to  help  management  do  its  job  of  coordination 
Computer  programming  is  but  one  of  the  many  elements  that  contribute  to 
the  development  and  operation  of  administrative  systems. 
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When  the  factors,  of  production  may  be  used  in  different  proportions,  that  is, 
when  they  may  be  considered  as  substitutes  for  one  another — the  propor'  ons 
of  each  productive  agent  required  to  produce  given  outputs  may  be  represented 
by  a  curve  such  as  that  shown  in  Figure  7  (and  labeled  the  best-practice 
production  function).  This  productive-agent  curve  may  be  expected  to  have  a 
shape  that  is  concave  to  the  origin  as  illustrated.*  It  repres  its  the  techni¬ 
cal  considerations  pertinent  to  the  production  process;  that  is,  any  point 
on  the  curve  represents  the  relative  equipment  and  manpower  costs  per  unit  of 
output  that  are  required  to  produce  that  output  within  existing  technological 
processes.  Computer  programs  are  but  one  part  of  this  process.  This  curve 
can  be  constructed  empirically  by  measuring  the  outputs  of  several  different 
systems,  and  the  manpower  a:*d  equipment  inputs  that  were  used  to  oroduee  these 
outputs.  Each  of  the  ten  dots  ir.  Figure  6  represents  a  different  system  (or 
organization,  or  firm)  whose  outputs  are  measured  in  the  same  units,  and  whose 
manpower  and  ADP  equipment  inputs  are  known.  The  line  ABCD  connects  the  points 
on  the  concave  hull  that  is  closest  to  the  origin.  That  is,  pairs  uf  points  are 
chosen  for  which  tb  line  joining  them: 

1.  Has  a  negative  slope. 

2.  Is  closer  to  the  origin  than  any  observed  point. 


*hased  on  the  proposition  that  the  greater  the  quantity  of  a  factor  use,  the 
less  Its  marginal  productivity  will  oe  '  ^ ) ■ 


Equipment /Output 
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Figure  7.  Construction  of  Indices  for  Evaluating 

Over-All  Administrative  System  Performance 
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To  complete  the  diagram,  broken  lines  are  drawn  parallel  to  the  axes  from 
the  extreme  points  of  the  concave  hull.  Firms  A,  B.  C,  ard  D  thus  represent 
the  productive  performance  that  is  meat  efficient  (minimum  use  of  resources 
per  unit  of  output)  for  their  own  particular  mix  of  resources;  theirs  are  the 
best  combinations  of  labor  and  capital  attained  by  any  of  the  firms  in  the 
sample.  The  reasons  why  one  firm  exhibts  better  performance  than  another  (such 
as  the  use  of  more  efficient  systems,  or  better  computer  programs)  are  not 
apparent  at  this  point — only  the  actual  differences  in  performance. 

The  function  constructed  in  Figure  7  is  an  approximation  of  the  best-practice 
production  function.  The  slope  of  the  production  function  at  any  point  indicates 
the  rate  of  substitution  of  labor  for  capital.  The  .lope  AB,  for  example, 
indicates  the  quantity  of  labor  that  must  be  substituted  for  capital  to  sustain 
a  constant  output  when  a  change  is  made  in  the  structure  of  the  firm  (i.e., 
in  the  mix  of  resources)  from  that  represer  ed  by  Firm  B  to  that  represented 
by  Firm  A. 

The  best-practice  production  function  is  a  technological  relationship,  portraying 
the  highest  state  of  the  art  attained  in  practice  by  any  member  of  the  sample, 
for  different  resource  mixes.  If  an  equal-cost  line--a  line  with  a  slope  equal 
to  the  un; i  cost  ratios  of  capital  and  labor--is  drawn  tangent  to  the  best- 
practice  produc'ior.  function,  the  point  of  tangency  repres-  .its  that  firm  that 
also  has  the  lowest-cost  combination  of  resources.  This  is  the  optimum  firm 
(Firm  B  In  the  example  of  Figure  7.)  Again,  this  does  not  mean  that  Firm  B  in 


Figure  7  has  the  best  computer  programs;  only  that  its  total  overall  operations. 
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Including  its  hardware  and  software  resources,  are  currently  producing  better 
results  at  less  cost  than  the  other  firms. 


The  construction  of  a  best-practice  production  function  in  the  manner  described 
above  implies  that  all  firms  on  the  best-practice  curve  are  operating  at  peak 
efficiency;  if  output  were  increased,  the  amount  of  resources  employed  would 
therefore  have  to  increase  correspondingly.  This  may  not  be  strictly  true, 
since  indivisibility  of  units  of  a  resource  may  provide  some  reserve  capacity. 
Also,  s  best-practice  firm  may  lead  the  field  to  such  an  extent  that  its  actual 
performance  substantially  understates  its  capability.  Such  considerations  mean 
that  the  proposed  method  tor  arriving  at  a  best-practice  production  function 
is  conservative  in  the  sense  of  producing  a  realistically  obtainable  target  for 
an  administrative  system  to  attain.  That  ui,  investment  in  system  changes  can 
be  expected  to  result  in  total  cost  savings  up  to  that  measured  by  the  difference 
between  current  operations  and  the  bcst-prsct ice  target. 

Having  constructed  a  production  function  representing  the  best-practice  standard 
for  a  sample  of  firms,  we  are  now  prepared  to  build  indices  relating  the  per¬ 
formance  of  any  giver  firm  to  this  standard. 


There  are  many  ways  of  comparing  the  performance  of 
standard  represented  by  a  best-practice  production, 
description  of  this  method  (19),  a  total  ot  twelve 
advanced;  at  this  time  I  will  illustrate  the  potent 


a  given  firm  with  the 
In  a  more  elaborate 
evaluation  indices  were 

iais  ov  desciibing  oniv 


two  ; 


the  Technical  Efficiency  index,  and  the  Relative  Manpower  '.9  Hi  ration  Index. 
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To  construct  the  Technical  Efficiency  Index  for  the  system  represented  in 
Figure  7  by  point  P,  point  P  is  connected  to  the  origin  by  a  straight  line 
Intersecting  the  best-practice  production  function  at  point  G.  The  technical 
efficiency  of  system  P  is  then  represented  by: 


Technical  Efficiency  Index  *  — 


This  index,  a  contribution  of  the  English  econott  ist ,  M.  J.  Farrell  (25)  >  measures 
the  success  of  system  P,  relative  to  a  hypothetic  i1  'inn  G  (firm  G  consists  of  a 
weighted  average  of  the  most  appropriate  observ'd  firms  on  the  best-practice 
production  function),  in  producing  maximum  output  from  a  giver.  9et  of  inputs 
Technical  efficiency  .  as  here  defined,  compares  systems  on  the  basis  of  equiva¬ 
lent  mix  of  inputs.  Much  of  the  difficulty  in  making  intersvstem  comparisons, 
as  between  systems  P  and  3  in  Figure  7 ,  arises  from  the  objection  that  such 
systems  are  so  fundamentally  different  that  a  comparison  wot  1 u  be  meaningless. 

We  have  previously  assumed  rough l v  equivalent  outputs;  the  technical  efficiency 
index  attempts  to  achieve  comparability  on  the  input  siue,  substantially 
mitigating  this  objection. 


Since  technical  efficiency  is  a  function  of  the  best  -pract Ice  proouct ion 
function,  the  evaluation  of  a  fir-  dene  .as  not  or.lv  on  that  firm's  realized 
ac:.  ievement  ,  but  or.  the  possible  acr  ievment  available  to  it  within  the 
constraints  of  technology .  Even  if  a  firm  : -proves  its  performance,  its 
technical  efficiency  index  will  decrease  if  even  greater  improvement  is 
achieved  by  other  firms  (this  would  be  represented  :v  a  g-eater  movement 
toward  tfu  ,-rigin  bv  the  production  function  in  Figure  '  t  nan  by  the  point 
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in  question).  Another  inportent  property  of  technicel  efficiency  is  that  the 
lowest-cost  firm  in  the  sample  will  elweys  heve  perfect  technicel  efficiency , 
but  so  nsy  e  number  of  other  firms  operating  with  different  proportions  of 
inputs. 

The  Relative  Manpower  Utilisation  Index  is  formed  (Figure  7)  by  drawing  a 
straight  line  from  point  P  parallel  to  the  manpower  expense  axis,  inter¬ 
secting  the  production  function  at  F  and  the  equipment-expense  axis  at  E.  The 
Relative  Manpower  Utilisation  Index  of  system  P  is  then  represented  by: 

EF 

Relative  Manpower  Utilisation  Index  “  — 

w 

This  index  is  a  measure  of  the  degree  to  which  manpower  expense  can  be  reduced 
(within  the  limits  of  the  technology)  assuming  that  equipment  expense  remains 
constant.  The  index  may  find  use  in  the  evaluation  of  current  manual  procedures 
and  various  personnel  factors.  It  is  superior  to  simply  comparing  labor  pro¬ 
ductivity  to  that  of  the  least-total-cost  firm  (point  B,  Figure  7)  or  the 
firm  with  the  lowest  manpower  expense  per  unit  output  (point  A,  Figure  7), 
because  it  explicitly  considers  the  contribution  of  the  equipment  resource 
(and  the  computer  programs  that  are  Included  with  it)  available  at  the 
pertinent  manpower /output  level. 

Both  the  Technical  Efficiency  Index  and  the  Relative  Manpower  Utilization 
Index  are  ratio  quantities.  And  there  is  always  some  question  as  to  whether 
ratio  measures  can  be  conceptually  adequate  measures  of  system  performance. 

Since  this  is  an  important  consideration  to  the  usefulness  of  the  technique 
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Just  described,  a  discussion  of  the  appropriateness  of  ratios  as  measures  of 
effectiveness  is  in  order. 

The  reason  often  advanced  for  avoiding  ratios  in  cost-effectiveness  studies  is 
that  optimal  ratios  like  optimal  computer  programs,  per  se,  are  usually  not  the 
primary  goal.  A  distraction  from  primary  objectives  can  lead  to  ridiculous  con¬ 
clusions.  For  example,  the  selection  of  a  house  on  the  basis  of  the  least  cost 
per  square  foot  could  result  in  the  choice  of  a  $500,000  mansion  rather -than  the 
$25,000  bungalow  that  may  more  closely  meet  the  real  needs  and  budget  of  the 
purchaser. 

Because  absolute  magnitudes  are  important,  the  general  statement  of  those 
criteria  appropriate  for  cost-effectiveness  studies  distinguishes  the  following  (19) 

1.  Fixed  gain,  variable  cost.  Resources  are  added  to  the  various  alter¬ 
natives  up  to  the  point  where  each  alternative  accomplishes  the 
objective;  the  best  alternative  is  that  with  the  least  absolute  cost. 

2.  Fixed  cost,  variable  gain.  The  alternative  is  chosen  that  accomplishes 
the  most  objectives  (or  the  greatest  degree  of  a  single  objective)  at 

a  given  cost. 

3.  Maximize  absolute  difference  between  gain  and  cost. 

Each  of  these  three  criteria  is  generally  acceptable;  each  has  its  advantages, 
depending  on  the  problems  of  measurement  and  the  circumstances  of  a  particular 
problem.  The  fixed-gain  case  is  especially  applicable  to  problems  in  which 
achievement  of  the  objective  is  binary— i.e.,  you  either  win  or  lose  (this 
would  be  appropriate  for  many  military  decisions).  Many  situations  in  which 
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gain  is  difficult  to  measure  may  be  handled  with  the  fixed-gain  (''just  meet 
the  specifications")  criterion.  fhe  fixed-cost  case  is,  of  course,  most 
useful  when  the  major  constraint  on  the  problem  is  a  fixed  budget;  it  can 
also  be  a  convenient,  cri  rion  when  there  are  many  subsidiary  objectives. 

Th.s  fixed-cost  and  the  fixed-gain  criteria  are  equivalent,  if  the  size  of 
either  gain  or  cost  is  tne  same  in  both  of  the  two  tests;  that  is,  if  the 
best  alternative  for  a  $100  budget  produces  a  gain  of  50,  then  the  least-cost 
choice  for  a  fixed-gain  of  50  would  be  the  some  alternative,  which  costs  $100. 
Therefore,  the  cl. ■'ice  between  the  fixed-cost  or  the  fixed-gain  criteria  depends 
largely  on  whether  cost  or  gain  can  be  more  readily  fixed  in  the  particular 
analysis  in  question. 

The  application  of  the  third  criterion — maxim! s 1  •'g  the  difference  between  gain 
and  cost — depends  on  the  ability  to  measure  gains  and  costs  in  the  same  kinds 
of  units.  This  criterion  is  the  same  as  the  familiar  business  criterion  of 
"profit  maximization."  Stated  in  the  economist's  terms  of  total  unit  costs 
and  outputs,  the  optimum  (maximum  gains)  occurs  at  the  output  level  at  which 
marginal  costs  equal  marginal  revenue.  When  opportunity  costs  are  considered — 
that  is,  when  the  gains  forfeited  by  not  choosing  an  available  alternative  are 
included  in  the  costs  of  the  remaining  alternatives  —  to  maximize  gai*.s  minus 
cost  is  the  same  as  maximizing  total  gains. 

The  reason  foi  the  above  elaboration  is  that  under  certain  circumstances,  a 
ratio  may  in  tact  be  simply  a  restatement  of  the  three  generally  acceptable 
criteria.  The  contention  is  that  for  most  business  firms,  profit  maximization 
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is  still  the  best  assumption  as  to  the  objective  of  the  firm,  and  that 
internal  efficiency  is  the  handmaiden  of  profit  maximization.  If  long-run 
unit  costs  do  not  necessarily  increase  with  increased  output  (i.e.  ,  if  there 
are  no  significant  diseconomies  of  large-scale  production),  and  there  is  good 
reason  to  believe  that  this  is  the  case  (2),  there  is  no  incentive  for  the  firm 
to  restrict  output;  on  the  contrary,  the  firm  would  tend  to  increase  its 
output  to  the  limits  of  the  market.  Under  these  conditions,  with  the 
objective  of  making  absolute  size  as  large  as  possible,  the  ratio  of  costs 
to  the  outputs  becomes  an  excellent  measure  of  efficiency;  it  is  equivalent 
to  either  the  minimum  cost  at  fixed  gain  o_r  the  maximum  gain  minus  cost 
criteria.  The  best  ratio  indicates  the  preferred  system,  no  matter  what 
the  scale. 

One  of  the  problems  in  constructing  indices  in  the  manner  suggested  herein 
is  that  it  is  not  always  easy  tc  find  situations  where  the  three  basic  requisites 
of  the  method  (objective  measure  of  output;  sample  of  systems  with  generally 
similar  outputs;  more  than  one  important  input  factor)  are  met.  However, 
there  are  many  cases  where  these  methods  do  apply.  The  potential  for  a 
50  percent  saving  in  manpower  expense  ir;  the  life  insurance  industry  cited 
earlier  in  this  paper  was  obtained  by  the  use  of  the  Relative  Manpower 
Utilization  Index.  And  frequently,  minor  variations  in  the  operations  of 
different  systems  Within  a  sample  can  be  accounted  for  with  such  standard 
techniques  as  multiple  regression  analysis.  These  matters  are  dealt  with  at 
length  elsewhere  (2,  19)* 
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Before  leaving  the  topic  of  evaluation  Indices,  one  further  point  is  in 
order.  This  relates  to  the  me-ning  of  such  overall  productivity  indices  as 
defined  herein  to  the  management  of  the  computer  programming  process.  The 
meaning  for  programming  management  is  simply  that  the  productivity  of  the 
system,  as  measured  by  the  Indices,  is  a  reflection  of  the  ultimate  success 
of  rhe  total  system;  computer  programs  are  merely  one  element  in  this 
productive  process.  Even  if  the  measurement  of  the  quality  efficiency 
of  computer  programs  poses  difficult  conceptual  and  empirical  problems, 
ultimate  system  productivity  can  be  quite  objective;  and  computer  programs 
have  little  value  if  they  do  not  have  an  impact  on  the  ultimate  objective  of 
the  system.  Thus  we  return  again  to  basic  management  principles,  in  this 
case  management  by  objectives  (2b),  to  develop  a  tool  for  system  evaluation. 
It  is  not  necessary,  however,  to  look  only  at  the  final  productivity  figures 
in  using  indices  such  as  those  suggested  here;  on  the  input  side,  the 
components  of  these  indices,  can  be  traced  back  to  their  source,  as 
illustrated  by  Figure  8.  Thus ,  a  series  of  related  indices  can  be 
constructed;  such  a  series  should  be  quite  useful  for  exploring  the  causes 
of  variations  in  total  systems  productivity. 


Figure  6.  One  Possible  Breakdown  of  Total  Administrative  Personnel  Expense 
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SUMMARY  AND  CONCLUSIONS 

I  have  ’-scussed  s--  aral  useful  tools  for  the  computer  programming  manager 
developed  at  the  System  Development  Corporation:  a  planning  guide;  empirically 
derived  computer  program  cost  estimating  guides;  a  project  reporting  and  a  data 
collection  system.  I  have  also  described  a  few  of  the  basic  concepts  of  a 
method  for  evaluating  the  comparative  efficiency  of  EDr  augment  administrative 
systems  with  measurable  outputs.  None  of  these  tools  are  revolutionary  or  novel 
in  their  design  or  application;  on  the  contrary,  their  development  illustrates 
the  application  of  well  established  management  principles. 

Both  the  planning  guide  and  the  cost  estimating  handbook  have  been  widely  dis¬ 
tributed  to  government  agencies  and  also  to  a  number  of  private  corporations. 

I  have  not  made  any  direct  effort  to  measure  the  benefits  to  these  organizations 
from  the  use  of  these  materials,  but  the  comments  received  to  dat*'  indicate  a 
gratifying  acceptance.  The  project  reporting  system  has  been  delivered  to  the 
Air  Force  Data  Systems  Design  Center  for  their  use.  And,  as  mentioned  earlier, 
the  productivity  evaluation  indices  have  been  successfully  applied  to  test 
several  management  hypotheses  for  a  sample  of  firms  in  the  life  insurance 
i ndustrv . 

The  experience  received  from  working  on  the  programming  management  tools  des¬ 
cribed  above  results  in  the  following  general  observe  * < nns : 

1.  Cost  data  should  be  collected  as  the  programming  project  proceeds,  not 
after  the  fact,  if  cost  prediction  with  accuracy  greater  than  that 
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demonstrated  by  the  material  presented  in  this  paper  is  desired. 

2.  To  develop  useiui  estimating  relationships  with  acceptable  predictive 
power,  the  analysis  should  have  a  comparatively  narrow  focus,  such  as 
on  specific  languages  and/or  applications.  For  example,  a  study  of 
UCBOL  programming  for  inventory  applications  could  be  of  considerable 
value,  both  as  a  research  vehicle  for  measuring  the  impact  of  important 
factors  (e.g.,  programmer  experience)  and  for  developing  accurate  and 
dependable  estimating  relationships. 

3.  All  permanent  programming  organizations  should  collect  cost  data  on  their 
own  operations.  This  would  enable  the  development  of  estimating  relation¬ 
ships  that  are  directly  pertinent  to  each  organization's  own  mix  of  resources, 
products,  and  particular  environment,  as  well  as  promote  better  project 
control.  The  extent  and  detail  of  the  data  collection  would  depend 

upon  the  particular  operations;  however,  the  determination  of  at  least 
the  total  resources  expended  er  project  Is  recommended  for  all  operations, 

4.  The  basic  structure  fcr  planning,  presenting  data  (the  handbooks),  or 
collecting  information  (the  reporting  system),  developed  by  SDC  could 
be  used  even  without  modification  by  all  computer  programming  organiza¬ 
tions.  Some  adaptation  of  this  material,  however,  would  probably  be 
advisable  (e.g.,  the  level  of  detail  at  which  costs  ate  collected  would 
depend  upon  the  size  of  programming  projects).  The  numerical  cost 
estimation  material  from  SDC's  research  (e.g.,  Table  I  and  Figures  ?,  4, 
and  3)  should  be  used  with  extreme  caution,  not  onlv  because  of  the 
validity  of  data  collected  ex-post-f actor  bv  questionnaire,  but  because 
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the  sample  studied  by  SDC  may  not  be  representative  of  the  program  mix 
or  the  user. 

5.  Evaluation  indices  ot  the  type  described  herein  are  empirically  work¬ 
able,  and  are  conceptually  adequate  measures  of  system  performance  for 
certain  kinds  of  systems.  They  are  recommended  for  use  in  research  on 
the  behavior  of  organizations  and  the  ultimate  impact  of  cc  iter 
programs.  Also,  the  construction  of  these  indices  and  the  tracing  of 
their  components  to  their  sources  could  be  used  as  a  foundation  for  a 
management  information  system. 

The  major  conclusion,  however,  is  the  basic  proposition  with  which  this  paper 
was  introduced:  that  the  most  expeditious  means  to  enhance  the  management  of 
computer  programming  today  is  to  apply  to  the  computer  programming  process  the 
principle  of  management  currently  successful  in  other  coordinated  activities. 
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13  ABSTRACT 


The  System  Development  Corporation  (SDC)  has  been  conducting  reseerch  in  the 
management  of  the  computer  programming  process  since  1963.  This  paper  briefly 
describes  some  of  the  results  of  this  research,  plus  individual  research  by  the 
author,  and  the  relevance  and  usefulness  of  the  results  to  the  operating  manager. 

The  paper  emphasizes  four  specific  types  of  management  tools  that  were  produced: 
(l)  planning  aids;  (2)  cost  estimating  guides;  (3)  a  project  reporting  and  control 
system;  (U)  a  technique  for  evaluating  the  effectiveness  of  certain  classes  of 
computer-centered  information  systems.  Each  of  these  tools  is  briefly  described, 
and  the  research  design  and  procedures  used  in  their  development  are  mentioned. 

The  experience  gained  —  at  SDC  and  elsewhere  —  in  applying  traditional  research 
techniques  to  the  computer  programming  process  has  yielded  certain  insights  regarding 
the  economic  and  management  dimensions  of  computer  programming,  and  several  of  these 
insights  are  discussed.  Foremost  among  this  author's  conclusions  is  that  the 
management  principles  that  apply  to  any  other  coordinative  activity  are  equally 
applicable  to  the  computer  programming  process.  Specific  techniques  may  differ, 
especially  at  the  lower  echelons,  but  these  differences  pertain  to  the  technical 
skills  and  procedures  of  the  production  process,  they  are  engineering,  not  management 
or  economic  issues.  The  paper  concludes  with  several  general  observations  pertinent 
to  future  research  in  computer  programming  economics  and  management,  and  the  use  of 
the  management  tools  described. 
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